Multimedia and multimodal systems: commonalities and differences

2. Definitions Due to differences on terminology usage in the literature the terms modality, medium and representation need to be defined. In a communication act, such as learning in a classroom, modality refers to the sensory or perceptual experience (e.g. visual, tactile, etc.) and is closely related to the individual. Medium is a means of conveying a representation (to a human), e.g. a diagram or a text. Representation sketches or stores information, e.g. semantic net, English language. Consider, for example, a classroom where pupils are taught about gravity. They listen and look while the teacher explains by speaking and gesturing (perceptual experience). The teacher has written an equation of gravity (w=m*g) on the board as well as a diagram. These are different types of representation. Some of the artefacts that the teacher or the pup ils use carry several representations, e.g. the board contains text, form or diagram written on it. Imagine now that pupils use the pen as an artefact to experiment with the law of gravity. Modality refers to the use of visual, auditory or tactile cues that pupils use to see the representations, hear the teacher’s presentation or handle the pen to understand by doing (sensory or perceptual experience).